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ABSTRACT 


In  most  policy  analysis  studios  there  are  a  large  number  of 
alternative  policies  and  a  large  number  of  impacts  to  be  considered. 

Time  and  budget  constraints  make  it  impossible  to  calculate  all  the 
impacts  for  all  of  the  alternatives.  As  a  result,  such  studies  include 
some  process  for  reducing  the  number  of  alternatives  to  be  examined  in 
detail.  The  process  is  often  implicit  and  non-sc ient i f ic  (e . g . ,  only 
the  decisionmaker's  three  favorite  alternatives  are  considered). 

s— M'his  paper  suggests  that  policy  analysis  studies  explicitly  include 
a  screening  step,  in  which  the  alternatives  to  be  examined  in  detail  and 
those  to  be  excluded  from  further  consideration  are  chosen  in  a 
scientific  and  systematic  manner.  The  output  from  this  step  is  a 
relatively  small  set  of  policy  alternatives  that  are  sufficiently 
attractive  that  they  deserve  a  more  thorough  evaluation.  Two  general 
screening  strategies  are  described.  They  are  illustrated  by  describing 
the.  screening  step  in  a  study  to  help  determine  an  overall  water 
management  policy  for  the  Netherlands. 
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I.  INTRODUCTION 


Policy  analysis  is  an  organized,  systematic  approach  to  problem 
solving.  It  can  be  used  to  study  the  problems  of  almost  any  system: 
health  systems,  political  systems,  industrial  systems,  urban  service 
systems,  etc.  Although  the  analyses  of  different  problems  often  show 
little  resemblance,  they  almost  always  involve  performing  the  same  set 
of  logical  steps.  By  now  these  steps  are  so  well  known  and  so  well 
documented  in  the  literature  that  they  are  referred  to  as  "the  classic 
steps  of  systems  analysis"  (see,  for  example,  (12,  p.  70J  and  [6,  p. 
10].)  The  steps  are  summarized  in  Fig.  1. 

In  any  large  policy  study  there  are  likely  to  be  numerous  measures 
of  performance  to  be  considered.  As  a  result,  the  enumeration  and 
evaluation  of  all  of  the  alternatives  as  called  for  in  Steps  4  and  5  of 
Fig.  1  will  be  impossible  to  carry  out  within  reasonable  time  and  budget 
constraints.  In  addition,  as  pointed  out  by  de  Neufville  and  Stafford 
(1,  p.  9]:  "Even  if  it  were  feasible  to  think  of  all  variations,  common 
sense  suggests  that  the  investigation  of  all  alternatives  is  not 
worthwhile:  some  are  not  sufficiently  different  to  warrant  separate 

treatment  for  each  and  some  are  clearly  dominated  by  others." 

The  situation  poses  a  problem  for  policy  analysts.  How  can  they 
carry  out  a  study  that  considers  the  full  range  of  alternatives  within 
their  time  and  budget  constraints?  In  most  cases,  they  solve  this 
problem  by  performing  an  incomplete  analysis.  They  ignore  some  impacts, 
examine  a  small  subset  of  the  alternatives,  or  both.  In  a  public  policy- 
study,  many  if  not  most  of  the  impacts  of  a  policy  will  be  hard  to 
quantify  (e.g.,  changes  in  the  quality  of  life)  or  hard  to  represent  in 
common  units  such  as  dollars  (e.g.,  the  loss  of  a  game  preserve  or  an 
increase  in  noise  pollution)-  The  usual  response  to  this  is  to  ignore 
all  but  a  few  of  the  impacts  in  performing  the  analysis.  This  is  the 
approacli  generally  taken  when  cost/benefit  analysis  or  optimization 
models  are  used  to  find  the  "best"  policy. 
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1.  Problem  identification 


Fig.  1  -  Steps  in  a  policy  analysis  study 
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A  second  solution  to  the  problem  that  makes  it  feasible  to  consider 
the  full  range  of  impacts  and  alternatives  is  to  split  Step  5  in  Fig.  1 
into  two  steps,  as  shown  in  Fig.  2.  In  Step  5a,  screening  models  would 
be  used  to  eliminate  the  least  promising  (or  to  select  the  most 
promising)  alternatives.  The  output  from  this  step  is  a  restriction  of 
the  set  of  alternatives  to  a  small  set  that  appear  to  be  promising-- 
i.e.,  are  sufficiently  sensible  and  beneficial  that  they  merit  a  more 
thorough  evaluation.  A  detailed  examination  of  the  remaining 
alternatives  (which  we  will  call  "impact  assessment")  would  then  be 
carried  out  in  Step  5b. 

Although  screening  is  implicitly  carried  out  in  most  large  policy 
analysis  studies,  it  is  usually  not  performed  in  a  rigorous  or 
systematic  way.  The  purpose  of  this  paper  is  to  point  out  that, 
although  a  policy  study  should  not  be  designed  to  find  the  "optimum" 
policy,  it  should  use  as  rational  a  process  as  possible  to  identify  good 
policies.  As  Manheim  [8,  p.  182]  expressed  it: 

"In  such  a  process,  we  try  to  find  as  good  a  solution  as 
possible,  balancing  the  prospects  of  improving  the  solution 
against  the  expenditures  of ...  resources  required  to  do  so." 

The  literature  on  screening  is  extremely  sparse.  Mood  [11,  p.  33] 
and  the  Handbook  of  Systems  Analysis  [10]  make  passing  reference  to  the 
concept.  Examples  of  the  use  of  screening  in  public  policy  studies  are 
contained  in  [2],  [3],  [4],  [5],  { 7  j ,  [9],  and  [13].  Although  the 
specifics  of  the  screening  will  depend  on  the  system  and  policy  problem 
being  analyzed,  there  are  two  basic  screening  strategies.  These  are 
described  in  Section  II.  In  Section  III  we  use  one  of  the  aforementioned 
studies  to  illustrate  the  two  strategies. 


Fig.  2  -  Addition  of  a  screening  step 
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II.  SCREENING  STRATEGIES 


Since  the  purpose  of  screening  is  to  narrow  the  range  of 
alternatives  needed  to  be  examined  in  detail,  its  value  increases  as  it 
becomes  easier,  cheaper,  or  faster  to  eliminate  alternatives.  Its  value 
decreases  as  it  becomes  less  discriminating,  less  reliable,  or  more 
costly.  There  are  certain  features  that  any  worthwhile  screening 
procedure  should  possess,  and  certain  universal  screening  rules  that 
apply  to  practically  every  situation.  Two  properties  of  good  screening 
prodedures  are:  (1)  no  very  good  alternative  will  be  missed,  and  (2) 
the  number  of  alternatives  to  be  evaluated  in  impact  assessment  will  be 
relatively  small.  Mood  (11,  p.  33]  and  Goeller  et  al .  [5,  p.  240] 
mention  several  general  screening  principles,  including: 

•  Infeasibility.  If  some  technical,  economic,  or  administrative 
difficulty  or  some  organizational  constraint  will  prevent 
implementation  of  an  alternative,  it  is  probably  not  worthwhile 
to  pursue  it. 

•  Political  unacceptability.  If  an  alternative  appears  to  be 
unacceptable  to  large  segments  of  the  public  or  to  the 
decisionmakers  who  must  ultimately  decide  on  its  adoption,  then 
it  might  as  well  be  left  out  of  consideration.  (In  some  cases, 
the  potential  benefits  from  the  alternative  might  be  so  large 
that  the  opposition  would  dissipate  when  the  analytical  results 
were  presented.  However,  this  is  a  rare  event.) 

•  Dominance.  If  two  alternatives  have  the  same  purpose  and  the 
benefits  from  one  are  at  least  as  good  as  the  benefits  from  the 
other  for  all  important  measures  of  performance,  then  the 
latter  can  be  dropped  from  further  consideration.  (Dominance 
is  usually  not  easy  to  find  or  to  show.) 
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Practically  all  structured  screening  procedures  employ  one  or  both 
of  the  following  "pure"  strategies: 

(1)  Use  Limited  Number  of  Impact  Measures .  This  strategy  starts 
with  a  large  number  of  well-defined  alternatives  and  eliminates 
unpromising  ones  (usually  one  at  a  time)  by  evaluating  their 
performance  on  a  few  key  measures.  Its  major  benefit  is  to 
speed  up  the  evaluation  of  well-defined  alternatives. 

(2)  Bound  Space  of  Promising  Alternatives.  This  strategy  eliminates 
sets  of  potential  alternatives  before  evaluating  any  specific 
ones  by  placing  constraints  on  the  characteristics  of  promising 
alternatives.  Its  major  benefit  is  to  focus  the  search  for  the 
well-defined  alternatives  to  be  evaluated. 

In  most  actual  studies  some  mixed  strategy  is  used.  The 
relationship  between  the  two  pure  strategies  is  similar  to  the 
relationship  between  primal  and  dual  linear  programs.  In  the  primal,  a 
sequence  of  feasible  solutions  is  evaluated.  The  dual  is  a  drive 
through  non-feasible  solutions  toward  the  feasible  space.  In  Strategy 
1,  a  set  of  well-defined  alternatives  is  evaluated.  Strategy  2 
constrains  the  space  in  which  "good"  alternatives  are  to  be  found. 

STRATEGY  1:  USE  A  LIMITED  NUMBER  OF  IMPACT  MEASURES 

The  essence  of  this  approach  to  screening  is  (1)  to  construct  an 
extensive  and  diverse  list  of  alternatives  that  may  or  may  not  turn  out 
to  be  promising,  and  (2)  to  produce  a  short  list  of  promising  tactics  by 
means  of  limited,  broad-brush  assessments  based  on  a  small  number  of 
impact  measures.  Manheim  refers  to  this  as  "sketch  planning"  [7,  p. 

585]  . 

As  an  illustration,  the  entire  process  can  be  compared  to  the  steps 
most  people  follow  in  buying  a  new  house.  A  broker  generally  has  too 
many  houses  listed  to  enable  a  potential  buyer  to  visit  each 
individually.  However,  a  tew  basic  criteria  such  as  neighborhood, 
purchase  price,  and  number  of  rooms  will  normally  reduce  the  number  of 
alternatives  significantly.  Then  more  detailed  criteria,  such  as  the 
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layout  of  rooms,  condition  of  plumbing,  etc.,  can  be  considered  for  the 
remaining  houses.  If  this  strategy  is  followed,  it  is  possible  that 
one's  "dream  house"  will  be  screened  out  in  the  first  stage  (e.g., 
because  the  price  is  slightly  too  high);  but  the  result  is  usually  quite 
satisfactory.  Figure  3  is  a  schematic  depiction  of  this  approach. 

Walker  and  Veen  (14)  applied  this  strategy  to  the  screening  of 
policy  options  (which  they  called  "technical  and  managerial  tactics") 
that  would  change  the  movement  and  storage  of  water  in  the  rivers, 
canals,  and  lakes  of  the  Netherlands.  The  tactics  were  designed  to 
alleviate  problems  caused  by  shortages  of  surface  water  and/or  by  the 
salinity  of  the  water.  Most  of  the  screening  was  performed  using  two 
measures:  the  tactic's  construction  cost  and  the  resulting  reduction  in 

the  country's  agricultural  losses.  A  more  complete  description  of  this 
screening  analysis  is  presented  in  Section  III. 

Similar  approaches  were  used  in  three  other  Rand  Corporation 
studies.  Two  of  them  ([2]  and  [3])  were  concerned  with  ways  to  reduce 
air  pollution.  The  studies  were  interested  in  analyzing  air  pollution 
control  alternatives  in  terms  of  a  wide  variety  of  environmental, 
transportat iona 1 ,  economic,  and  distributional  impacts.  However,  in 
each  study  the  number  of  feasible  alternatives  was  so  vast  that  it  was 
impractical  to  evaluate  all  of  them  in  terms  of  detailed  impacts. 
Alternatives  were  screened  after  calculating  only  their  costs  and 
effectiveness  in  reducing  emissions. 

The  third  Rand  Study  [4]  examined  the  possible  consequences  of 
alternative  ways  of  protecting  the  Oosterschelde ,  an  estuary  in  the 
southwestern  Netherlands,  against  North  Sea  flooding.  The  impacts 
considered  in  the  analysis  included  the  security  of  people  and  property 
from  flooding;  the  financial  costs  of  constructing  and  operating 
protective  facilities;  the  changes  in  the  ecology  of  the  region,  the 
economic;  effects;  the  water  management  impacts;  and  the  various  social 
effects,  including  the  displacement  of  households.  The  screening 
analysis  concentrated  on  a  few  impacts  related  to  security,  ecology,  and 
construction  costs.  For  example,  any  alternative  that  would  provide  the 
desired  level  of  security  with  a  higher  cost  and  a  less  desirable 
ecology  than  some  other  alternative  was  ruled  out. 
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Choice  of 
policv 


Fig.  3  —  A  screening  strategy  that  eliminates 
unpromising  alternatives 
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STRATEGY  2:  BOUND  THE  SPACE  OF  PROMISING  ALTERNATIVES 

The  essence  of  this  strategy  is  to  place  constraints  on  the 
characteristics  that  promising  alternatives  must  possess.  After 
screening,  there  would  be  very  few  a  1 ternat ives  left  to  evaluate  because 
very  few  could  be  defined  that  would  possess  these  characteristics.  The 
screening  process  itself  may  end  in  the  identification  of  a  single 
promising  alternative,  or  the  step  after  screening  might  be  to  identify 
the  feasible  alternatives  (if  any)  within  the  restricted  decision  space 
(see  Fig.  4)  . 

The  art  in  this  strategy  lies  in  specifying  the  constraints  on  the 
alternatives.  Many  of  the  constraints  are  likely  to  be  quite  arbitrary. 
However,  if  the  constraints  are  specified  well,  the  strategy  can  be  very 
efficient.  Each  constraint  can  eliminate  entire  sets  of  alternatives. 
For  example,  if  all  promising  alternative  highway  routes  between  City  A 
and  City  B  had  to  pass  through  City  C,  large  numbers  of  possible  routes 
could  be  eliminated  immediately. 

Manheim  [8]  provides  a  highly  structured  example  of  the  use  of  this 
strategy.  He  presents  an  optimal  search  branch-and-bound  algorithm  that 
concludes  with  a  restricted  decision  space  containing  a  single 
alternative.  The  specific  application  is  highway  route  location.  Full 
specification  of  a  single  alternative  may  require  assigning  values  to 
over  30,000  variables.  However,  in  his  approach,  the  location  process 
passes  through  several  stages.  In  each  successive  stage,  the  degree  of 
detail  with  which  the  alternatives  are  specified  is  increased,  so  that, 
only  in  the  final  phase  (called  "final  design")  are  the  specific  values 
of  all  the  variables  assigned. 

This  strategy  for  screening  can  be  compared  to  the  approach 
commonly  used  in  designing  a  building.  The  process  starts  with  an 
evaluation  of  alternative  functional  requirements  for  the  building. 

There  may  be  many  possibilities.  In  this  step  all  but  a  few  are  ruled 
out.  This  is  then  followed  in  sequence  by  the  specification  of  building 
requirements,  the  drawing  up  of  general  plans  for  the  building,  and  the 
drawing  up  of  detailed  blueprints.  At  each  step,  large  numbers  of 
possible  blueprints  are  being  eliminated  without  their  having  to  be 


drawn . 
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Fig.  4  —  A  screening  strategy  that  bounds 
the  space  of  promising  alternatives 
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Variations  on  this  strategy  have  been  used  in  several  policy 
studies  besides  Manheim's.  Walker  and  Veen  [14]  applied  it  as  a  first 
step  in  their  screening  of  technical  and  managerial  tactics  for  water 
management  in  the  Netherlands.  They  calculated  an  upper  bound  on  the 
expected  reduction  in  agricultural  losses  that  tactics  could  produce  in 
each  of  the  eight  regions  of  the  country.  The  upper  bound  was  so  small 
in  two  of  the  regions  that  no  tactics  pertaining  to  these  regions  were 
evaluated.  (See  Section  III.) 

Martland  et  al.  [9,  p.  14]  report  that  the  U.S.  Department  of 
Transportation  (DOT)  used  this  strategy  to  determine  which  parts  of  the 
railroad  system  in  the  Midwest  and  Northeast  Region  should  be  subjected 
to  detailed  scrutiny  to  see  if  there  was  excess  capacity.  Using  a 
statistical  study  of  branch  line  operations,  DOT  was  able  to  determine 
the  relationship  between  a  branch's  profitability  and  the  independent 
variables  line  length  and  traffic  volume.  As  a  result,  it  could 
identify,  for  branch  lines  of  any  length,  the  range  of  traffic  volumes 
that  would  be  potentially  unprofitable.  After  analyzing  every  branch 
line  in  the  region  using  this  criterion,  DOT  found  that  25  percent  of 
the  mileage  was  "potentially  excess."  It  then  concluded:  "These  are 
the  lines  which  should  receive  the  closest  analysis  in  the  process  of 
restructuring  the  local  rail  service  network." 

Rogers  and  Berry  [13]  used  a  similar  idea  to  find  urban  telephone 
feeder  routes  whose  length  and  population  distribution  make  a  pair  gain 
system  (PGS)  installation  economically  feasible.1  For  a  route  of  given 
length  and  population  distribution,  their  screening  model  defines  the 
set  of  locations  at  which  a  PGS  installation  is  cheaper  than  using  only 
additional  copper  wire  pairs  to  meet  subscriber  growth. 


1  In  a  PGS  installation,  the  signals  from  many  copper  wire  pairs 
(individual  subscribers'  lines)  are  concentrated  and  multiplexed  onto 
many  fewer  pairs. 


III.  SCREENING  IN  PAWN 


OVERVIEW 

In  1976  a  severe  drought  cost  the  Netherlands  more  than  $2  billion 
in  agricultural  losses.  In  addition,  low  river  levels  caused  serious 
shipping  delays  and  substantial  economic  losses  because  ships  could  not 
navigate  the  waterways  with  full  loads,  and  the  water  shortage  worsened 
water  quality  problems.  In  1977,  the  Dutch  government  commissioned  a 
broad  study  of  the  country's  water  management  system.  The  resulting 
project,  called  Policy  Analysis  of  Water  Management  for  the  Netherlands 
(PAWN),  was  a  3-year,  125  man-year  effort  conducted  jointly  by  the  Rand 
Corporation,  the  Ri jkswaterstaat  (the  government  agency  responsible  for 
water  control  and  public  works),  and  the  Delft  Hydraulics  Laboratory  (a 
Dutch  research  organization.)1  The  objectives  of  PAWN  were  twofold: 

(1)  to  develop  a  methodology  for  assessing  the  multiple 
consequences  of  water  management  policies,  and 

(2)  to  apply  this  methodology  to  evaluate  a  number  of  alternative 
water  management  policies. 

The  study  considered  a  wide  range  of  consequences  of  the  water 
management  policies  (the  consequences  were  called  "impacts").  Impact 
measures  were  defined  to  cover  the  objectives  of  all  affected  groups 
(e.g.,  farmers,  shippers,  industrial  firms,  the  country's  ecology, 
public  health,  etc.),  and  to  reflect  both  equity  and  efficiency 
considerations.  Table  1  lists  a  subset  of  the  measures  used  by  PAWN  in 
assessing  the  impacts  of  alternative  water  management  policies. 


1  Results  of  the  study  have  been  documented  in  a  series  of  21 
publications.  Goeller  et  al.  [5]  provide  an  overview  of  the 
methodology  and  a  summary  of  the  results.  Walker  and  Veen  [14]  provide 
details  of  the  screening  analysis. 
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Table  1 

PAWN'S  IMPACT  ASSESSMENT  MEASURES 


Impacts  on  Water  Management  System 

Costs  of  tactic 
Risk  of  flooding 

Direct  Impacts  on  Users 

1.  Agriculture 
Gross  benef its 

By  crop  (13  crops) 

By  region  (8  regions) 

Sprinkler  costs  (fixed  and  variable) 

Net  benefits  (by  who  receives  benefits) 

Dutch  (producers,  consumers,  government) 

Foreign  (producers,  consumers,  government) 

2.  Shipping 

Low  water  losses  (Dutch  and  foreign) 

Lock  delay  losses  (Dutch  and  foreign) 

Annualized  cost  of  changing  fleet  (Dutch  and  foreign) 

3.  Power  Plants 

4.  Industrial  Firms  and  Drinking  Water  Companies 
Revenues  of  drinking  water  companies 

From  commercial  firms 
From  industrial  firms 
From  households 

Cost  of  supplying  drinking  water 

The  percentage  of  groundwater  in  drinking  water 

Total  industrial  water  consumption  (by  source) 

Industrial  cost  increases  (taxes,  drinking  water  costs,  etc.) 

5.  Other  users 

Lock  delays  for  recreational  boats 
Miles  of  fresh  water  beaches 

Impacts  on  the  Environment 

Violations  of  water  quality  standards  (salinity,  chromium,  BOD, 
phosphates,  heat) 

Changes  in  groundwater  levels 
Algae  growth 

Damage  to  nature  from  constructing  facilities 

Impacts  on  Entire  Nation 

Jobs 

Sales 

Trade 

Public  health 
Government  revenues 


A 


14 


A  water  management  policy  involves  a  mix  of  tactics,  each  a  single 
action  to  affect  water  management,  such  as  building  a  particular  canal 
or  taxing  a  particular  use.  Four  kinds  of  tactics  were  distinguished  in 
PAWN: 

(1)  Technical  tactics,  which  add  to  or  modify  the  current  water 
management  infrastructure  (e.g.,  dig  a  new  canal,  enlarge  a 
pumping  station,  or  build  a  dike) 

(2)  Managerial  tactics,  which  change  the  rules  by  which  a 
particular  infrastructure  is  operated  (e.g.,  change  the 
operation  of  weirs,  pumps,  and  sluices  so  as  to  affect  the 
water  level  in  lakes  or  the  flow  in  rivers  and  canals) 

(3)  Pricing  tactics,  which  impose  a  charge  on  water  use  or 
discharge . 

(4)  Regulation  tactics,  which  control  water  use  or  discharge  with 
legal  and  administrative  measures. 

The  first  two  kinds  of  tactics  directly  affect  the  supply  of  water  to 
particular  users;  the  last  two  directly  affect  users'  demand  for  water. 
Different  screening  approaches  were  used  for  the  first  two  and  last  two 
kinds  of  tactics.  In  the  remainder  of  the  paper  we  describe  the 
screening  of  technical  and  managerial  tactics. 

ESTIMATION  OF  COSTS  AND  BENEFITS 

Tactics  were  determined  to  be  promising  or  unpromising  based  on  a 
comparison  of  their  costs  and  benefits.  The  cost  assigned  to  a  tactic 
was  its  annualized  fixed  cost  (AFC).2  Operating  costs  (which  were 
mainly  the  cost  of  electricity  to  run  pumps)  were  ignored  because  they 
were  very  small  relative  to  the  fixed  costs. 

The  benefits  and  disbenefits  from  alternative  water  management 
policies  accrue  primarily  to  the  direct  users  of  water:  farmers, 
shippers,  industries,  and  drinking  water  companies.  The  policies  also 

2  A  tactic’s  AFC  was  obtained  by  applying  a  capital  recovery  factor 
of  0.10  to  the  investment  cost  and  adding  the  fixed  annua]  operating 
cost.  A  capital  recovery  factor  of  0.10  reflects  a  useful  life  of 
approximately  50  years  and  a  discount  rate  of  10  percent. 
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have  more  global  impacts,  such  as  on  consumers,  the  environment,  and 
other  countries.  In  screening,  attention  was  focused  on  agriculture, 
since  it  is  the  largest  consumer  of  water,  and  it  suffers  most  from 
water  shortages  and  salinity.  The  use  of  water  by  agriculture  is  about 
9  times  that  of  the  drinking  water  companies  and  50  times  that  of 
industries.  In  1976  it  is  estimated  that  agriculture  suffered  losses  of 
over  6  billion  guilders  due  to  water  shortages  and  about  500  million 
guilders  due  to  the  salinity  of  the  water  used  to  irrigate  crops.3  The 
impacts  of  the  tactics  on  shipping  (which  is  affected  by  low  water 
levels  on  the  nation's  waterways)  were  also  considered. 

The  benefits  from  a  tactic  are  defined  as  the  difference  between 
the  losses  to  agriculture  and  shipping  if  the  tactic  is  implemented  and 
the  losses  to  those  user  categories  without  the  tactic.  The  major 
categories  of  losses  that  were  considered  in  the  screening  analysis  are: 

•  Agriculture  shortage  losses.  If  a  tactic  increases  the  supply 
of  water  to  an  area,  then,  in  years  with  low  rainfall,  a  small 
proportion  of  the  crops  will  die  due  to  an  insufficient  amount 
of  water.  The  benefits  from  increased  supply  are  therefore 
measured  in  terms  of  reduced  agriculture  shortage  losses. 

•  Agriculture  salinity  losses.  If  a  tactic  is  able  to  reduce  the 
salinity  of  water  supplied  to  agriculture,  then  a  smaller 
proportion  of  the  crops  will  be  damaged  by  salt.  The  benefits 
from  reduced  salinity  are  therefore  measured  in  terms  of 
reduced  crop  losses  due  to  salinity  damage. 

•  Low  water  shipping  losses.  If  the  depths  of  the  country's 
waterways  are  not  sufficient,  ships  cannot  carry  their  maximum 
loads,  but  must  travel  with  less  cargo  to  reduce  their  drafts. 
This  means  more  trips  and  higher  operating  costs  for 
transporting  a  given  amount  of  goods.  Tactics  that  change 
flows  and  water  levels  on  certain  waterways  will  affect 
shipping  depths.  The  benefits  from  these  tactics  are  measured 
in  terms  of  a  reduction  in  shipping  losses. 
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A  guilder  is  worth  approximately  $0.33. 
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The  benefits  from  a  tactic  vary  from  year  to  year  depending  on  the 
rainfall  and  river  flows.  For  example,  tactics  designed  to  reduce 
shortage  losses  will  produce  higher  benefits  in  dry  years  than  in  wet 
years.  Thus,  since  river  flows  and  rainfall  vary  from  year  to  year,  the 
benefits  of  a  tactic  for  any  year  are  unpredictable.  To  compare  a 
tactic's  benefits  to  its  cost,  an  estimate  of  its  expected  benefits  (the 
average  over  many  years)  was  needed.  The  approach  used,  which  is 
described  below,  produces  upper  and  lower  bounds  on  the  expected  annual 
benefits  by  taking  weighted  averages  of  the  benefits  in  four  years  that 
had  very  different  rainfall  and  river  flow  patterns  (1943,  1959,  1967, 
and  1976). 

Based  on  the  losses  to  agriculture  for  the  years  1933  through  1976, 
it  was  estimated  that  both  the  shortage  and  salinity  losses  associated 
with  1976  would  be  exceeded  only  one  year  in  44,  or  in  approximately  2 
percent  of  all  years.  The  exceedence  probabilities  for  the  other  three 
years  were  also  calculated.  The  results  for  the  four  "external  supply 
scenario"  years  are  given  in  Table  2. 

To  obtain  the  upper  bound  for  expected  annual  benefits,  it  was 
assumed  that  the  benefits  in  1976  would  be  obtained  in  all  years  drier 
than  1959;  the  benefits  in  1959  would  be  obtained  in  all  years  between 
1959  and  1943  in  dryness;  etc.1*  For  the  lower  bound  it  was  assumed  that 

Table  2 

PROBABILITIES  OF  ANNUAL  LOSSES  EXCEEDING 
THOSE  OF  FOUR  SCENARIO  YEARS 


1976 

1959 

1943 

1967 

Shortage  losses 

.02 

.07 

.21 

.63 

Salinity 

.02 

.09 

.  13 

.57 

*  Treating  a  tactic  as  promising  if  its  annualized  fixed  cost  is 
less  than  the  upper  bound  on  its  expected  annual  benefits  is  therefore 
very  conservative.  A  year  as  bad  as  1976  is  treated  as  if  it  is 
expected  to  occur  once  every  11  or  14  years  instead  of  once  every  50 
years.  So  only  very  poor  tactics  will  be  screened  out. 


the  1976  benefits  would  be  obtained  in  all  years  drier  than  1976;  1959 
benefits  would  be  obtained  in  a  year  between  1976  and  1959  in  dryness, 
etc.;  it  was  also  assumed  that  there  would  be  no  benefits  from  the 
tactics  in  any  year  wetter  than  1967. 

Letting  B(y)  be  the  benefits  obtained  in  year  y  from  implementing  a 
tactic,  the  formulas  for  upper  and  lower  bounds  on  the  expected  annual 
benefits  (LAB)  are: 

For  shortage  losses:  .02B(1976)  +  .05B(1959)  +  .  14B(1943) 

+  . 42B (  1967 )  <  EAB  <  .07B(1976)  +  .  14B ( 1959  ) 

+  ,42B(  1943)  +  . 37B (  1967 ) 

For  salinity  and  .02B(1976)  +  ,07B(1959)  +  .04B(1943) 

shipping  losses:  +  ,44B(1967)  <  EAB  <  .09B(1976)  +  .04B(1959) 

+  .  44B  (  1943)  +  . 43B (  1967 ) 

Estimation  of  the  benefits  for  each  of  the  four  years  was  carried 
out  using  a  model  called  the  Distribution  Model,  which  simulates  the 
major  components  of  the  surface  water  system  of  the  Netherlands  [15]. 
Given  information  over  time  on  how  much  water  enters  the  country  (from 
rivers,  rainfall,  and  groundwater  flows),  and  how  much  water  is 
extracted  by  the  various  user  groups,  the  model  calculates  the  water 
flows  in  the  major  rivers  and  canals,  the  levels  of  the  lakes,  and  the 
concentration  of  salt  in  these  waters.  Using  this  information,  it 
calculates  the  agricultural  shortage,  and  salinity  losses,  and  losses  to 
shipping  from  low  water  flows  and  lock  delays. 

SCREEN  1:  BOUNDING  THE  SPACE  OF  PROMISING  ALTERNATIVES 

For  purposes  of  the  analysis,  the  Netherlands  was  divided  into 
eight  regions  (see  Fig.  5).  The  general  approach  to  screening  technical 
and  managerial  tactics  was  to  prepare  a  list  of  potential  tactics  for 
each  region  and  to  evaluate  each  one  using  a  small  subset  of  the  impact 
measures.  However,  before  preparing  these  lists,  a  pre-screening 
analysis  (Screen  1)  was  conducted  to  identify  those  regions  and  scenario 


Fig.  5  -  FAWN  analytic  regions 
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assumptions5  for  which  the  costs  of  implementing  tactics  were  unlikely 
to  be  offset  by  the  expected  annual  benefits  in  terms  of  reduced  crop 
losses.  These  regions  could  then  be  ignored  in  the  subsequent  analysis 
(except  if  they  had  serious  problems  with  respect  to  other  impact 
measures  )  . 

For  each  region  and  several  demand  and  supply  scenarios,  estimates 
were  made  of  the  agriculture  shortage  losses  that  would  occur  (1)  with 
the  existing  water  management  infrastructure,  and  (2)  if  the 
infrastructure  were  expanded  so  that  there  were  no  shortage  losses  in 
areas  with  access  to  surface  water  (i.e.,  the  demands  for  irrigation 
water  were  fully  met). 

The  difference  between  the  two  estimates  of  shortages  losses  (which 
was  called  the  preventable  losses)  would  be  the  maximum  benefits  that 
could  be  expected  from  any  tactics  designed  to  reduce  water  shortages  in 
the  region.  An  upper  bound  on  the  preventable  losses  was  obtained  by 
making  use  of  the  formula  for  shortage  losses  presented  above.  Letting 
L(y)  be  the  preventable  losses  in  year  y,  an  upper  bound  on  the  expected 
annual  preventable  losses  (UB)  is  given  by: 

LB  =  . 07L( 1976)  +  . 14L(1959)  +  ,42L(1943)  +  .37L(1967). 

Table  3  presents  the  maximum  benefits  by  region  for  two  demand 
scenarios--a  low  demand  scenario  (corresponding  to  the  current  use  of 
water  for  irrigation)  and  a  high  demand  scenario  (representing  projected 
future  demand).  The  results  show  that  unless  the  demand  for  irrigation 
water  increases  substantially,  preventable  shortage  losses  are  expected 
to  be  small,  so  few  if  any  tactics  are  needed.  Thus,  the  next  stage  of 
the  screening  analysis  (Screen  2)  was  limited  to  the  high  demand 
scenario,  except  for  tactics  involving  Region  8.  In  addition,  since  the 
maximum  benefits  for  Regions  3  and  6  were  very  low  in  both  scenarios,  no 
technical  tactics  for  reducing  agriculture  shortage  losses  in  these  two 
regions  were  analyzed. 

5  A  scenario  consisted  of  factors  that  are  uncertain  and  whose 
value  will  be  determined  by  outside  processes  (political  or  natural) 
beyond  the  control  of  water  management  decisionmakers.  The  scenario 
variables  included  the  amount  of  sprinkler  equipment  that  would  be 
installed  by  farmers,  and  the  amount  of  salt  that  would  be  dumped  into 
the  Rhine  River  outside  the  Netherlands. 
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Table  3 

UPPER  BOUNDS  ON  EXPECTED  ANNUAL 
PREVENTABLE  LOSSES 

(In  Millions  of  Guilders) 
Region  Low  Demand  High  Demand 


1  0.0  7.5 

2  0.6  19.9 

3  0.0  0.2 

4  0.0  0.7 

5  0.8  4.2 

6  0.0  0.3 

7  0.0  0.5 

8  3.7  28.1 


SCREEN  2:  USING  FEW  IMPACT  MEASURES 

The  second  phase  of  the  screening  analysis  began  with  the 
compilation  of  a  list  of  technical  and  managerial  tactics  that  were 
designed  to  resolve  the  national  and  regional  shortage  and  salinity 
problems.  The  major  sources  of  tactics  were  reports  prepared  by  the 
Ri jkswaterstaat  and  discussions  with  Dutch  water  management  experts.  In 
addition,  the  results  of  Screen  1  led  to  the  identification  of  new 
tactics  aimed  at  reducing  agriculture  shortage  losses  in  Regions  2  and 
8,  where  the  preventable  losses  were  particularly  high.  A  total  of  57 
tactics  were  analyzed  in  Screen  2. 

The  Distribution  Model  was  used  to  estimate  the  expected  annual 
benefits  of  each  of  the  tactics.  The  model  was  run  four  times  for  each 
of  the  tactics--once  for  each  supply  scenario  (1976,  1959,  1947  and 
1967).  Then  the  formulas  given  above  were  used  to  obtain  upper  and 
lower  bounds  on  the  expected  annual  shortage  losses,  salinity  losses, 
and  shipping  losses.  The  losses  were  summed  to  obtain  the  expected 
annual  benefits  (EAB),  which  were  then  compared  to  the  tactic's 
annualized  fixed  cost  (AFC).  If  the  AFC  exceeds  the  upper  bound  on  the 
EAB,  the  tactic  is  clearly  not  promising.  If  the  AFC  is  less  than  the 
lower  bound  on  the  EAB,  the  tactic  is  clearly  worth  further 
consideration.  It  was  decided  to  pass  along  to  the  impact  assessment 
phase  all  tactics  whose  AFC  was  less  than  the  upper  bound  on  the  EAB, 


recognizing  that  the  expected  annual  benefits  might  in  fact  turn  out  to 
be  less  than  the  annualized  fixed  cost. 

To  illustrate  how  the  Screen  2  analysis  was  performed,  the  analysis 
of  one  of  the  tactics  is  summarized  below.  The  tactic  analyzed  involves 
building  a  canal  to  reduce  agriculture  salinity  losses  in  Region  5. 

Some  of  the  country's  most  valuable  crops  are  grown  in  this  region-- 
the  world  renowned  Dutch  bulbs  (tulips,  hyacinths,  etc.),  greenhouse 
vegetables  (mainly  tomatoes,  cucumbers,  and  lettuce),  and  greenhouse 
flowers  (mainly  roses,  carnations,  and  chrysanthemums).  All  are  quite 
sensitive  to  salt.  The  principal  inlet  point  for  the  region's  surface 
water  is  located  along  the  Hollandsche  IJssel  at  Gouda  (see  Fig.  6). 

The  proposed  solution  for  this  problem  was  to  extract  water  for  the 
region  at  a  point  further  upstream,  where  there  was  no  danger  of 
contamination  from  North  Sea  salt  water.  The  water  would  be  transported 
to  Gouda  via  a  new  canal  to  be  built  through  the  portion  of  the  country 
known  as  the  Krimpenerwaard  (see  Fig.  6). 

The  benefits  from  the  canal  were  measured  by  the  reduction  in 
agriculture  salinity  losses  in  Region  5.  These  benefits  are  partially 
offset  by  increases  in  shipping  losses  caused  by  reduced  flows  on  one  of 
the  country's  major  rivers,  which  would  supply  water  to  the  canal. 
Distribution  Model  runs  were  made  with  and  without  the  Krimpenerwaard 
Canal  using  the  high  demand  scenario  and  the  1976,  1959,  and  1943 
external  supply  scenarios.  (No  year  wetter  than  1943  was  run  because  it 
was  clear  that  the  canal  would  not  be  needed  in  such  years.)  The 
differences  between  the  salinity  and  shipping  losses  in  the  two  sets  of 
runs  (with  and  without  the  canal)  were  used  as  the  benefits  to  be 
derived  from  building  the  canal.  Table  4  presents  the  results  derived 
from  these  runs,  as  well  as  the  upper  and  lower  bounds  calculated  from 
them . 

The  numbers  in  Table  4  provide  several  important  insights.  First, 
North  Sea  salt  is  a  serious  problem  only  in  an  extremely  dry  year  such 
as  1976.  Second,  the  increased  losses  to  shipping  from  this  tactic  are 
significant.  In  fact,  in  1959  and  1943  they  outweigh  the  reductions  in 
salinity  losses  that  the  tactic  achieves.  Finally,  the  upper  bound  on 
the  tactic's  expected  annual  benefits  is  less  than  its  annualized  fixed 
cost,  which  means  that  it  is  not  a  promising  tactic.  It  was  therefore 


Fig.  0  -  Rout*  of  K'li 
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screened  out--i.e.,  not  examined  further  in  later  stages  of  PAWN. 

SCREENING  RESULTS  AND  CONCLUSIONS 

Screening  in  the  PAWN  project  was  a  simple  and  inexpensive,  yet 
powerful  way  to  reduce  the  effort  that  might  have  been  required  to 
evaluate  many  potentially  valuable  alternatives  against  scores  of  impact 
measures.  The  expressed  purpose  of  screening  was  to  select  from  the 
large  number  of  potential  tactics  a  reasonably  small  number  that  merited 
a  more  detailed  evaluation.  This  objective  was  achieved.  But  screening 
accomplished  more  than  just  producing  intermediate  results  to  be  used  as 
inputs  to  the  next  phase  of  the  project.  It  produced  insights  that  have 
helped  to  guide  subsequent  water  resources  planning  in  the  Netherlands. 

First,  Screen  1  showed  that  there  are  relatively  few  tactics  for 
reducing  agriculture  shortage  losses  that  arc  worth  considering  further, 
unless  the  demand  for  surface  water  were  to  increase  significantly  above 
current  levels.  It  also  showed  that  it  would  not  ba  worth  the  effort  to 
generate  or  evaluate  any  alternatives  for  reducing  shortage  losses  in 
two  of  the  eight  regions  of  the  country. 

The  insights  from  Screen  2  were  even  more  valuable  and  somewhat 
surprising.  Prior  to  the  PAWN  study,  the  Dutch  had  been  considering 
several  large,  expensive  construction  projects.  Almost  all  of  these 

Table  4 

COMPARISON  OF  BENEFITS  AND  COSTS  OF  BUILDING 
A  KRINPENERWAARD  CANAL 

(millions  of  guilders) 

Expected 
Annual  Benefits 

-  Annualized 

Upper  Lower  Fixed 

Bound  Bound  Cost 


Salinity 

41.5 

0.2 

0.0 

0.0 

Shipping 

(6.9) 

(2.4) 

(0.3) 

0.0 

Net 

34.7 

(2.2) 

(0.3) 

0.0 

External  Supply  Scenario 

Type  of 

Benefit  1976  1959  1943  1967 


2.9 


0.5 


5.7 
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projects  were  screened  out  because  the  costs  of  construction  were 
greater  than  the  most  optimistic  assessment  of  the  potential  benefits. 
Savings  to  the  Dutch  government  from  not  proceeding  with  those  projects 
are  estimated  in  the  hundreds  of  millions  of  dollars. 

Except  for  a  few  very  inexpensive  national  tactics,  most  of  the 
promising  tactics  primarily  affect  a  single  region.  This  suggested  that 
the  national  water  management  infrastructure  is  functioning  rather  well, 
and  that  future  water  management  analysis  and  policymaking  should  be 
focused  more  on  regional  water  management  problems. 

Nine  inexpensive  regional  construction  projects  were  shown  to  be 
highly  cost-beneficial.  These  tactics,  together  with  five  national 
tactics,  were  passed  on  to  the  impact  assessment  phase  of  the  project. 
Six  of  these  fourteen  tactics  were  found  to  be  promising,  independent  of 
the  demand  scenario  used.  They  include  two  aimed  at  reducing 
agriculture  salinity  losses,  one  that  will  help  reduce  agriculture 
shortage  losses,  two  that  will  reduce  low  water  shipping  losses,  and  one 
that  is  designed  to  reduce  the  chance  of  flooding.  The  total  annualized 
fixed  cost  for  all  six  tactics  is  less  than  $3  million.  For  the  high 
demand  scenario,  eight  more  tactics  were  found  to  be  very  attractive. 
Eleven  others,  although  promising,  are  dominated  by  one  of  those  eight 
(i.e.,  they  attack  the  same  problem,  but  have  higher  costs  and/or  lower 
bcnef its) . 
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